Eecient Top-down Jacobian Evaluation of Tree-structured Neural Networks

Tree-structured neural networks (TSNN) are known to be universal ap-proximators that have easily derivable equivalent implementations as feed-forward neural networks with two hidden layers. What makes them particularly interesting for large-scale and real-time applications such as adaptive control is their ability to support eecient lazy evaluation by exploiting their hierarchical structure and a special kind of sigmoid function. A new and highly time and space eecient algorithm is presented for the evaluation of the Jacobian matrix of a given TSNN function, which is extremely helpful for any kind of search in the input space such as function inversion. The proposed algorithm diiers from earlier approaches in that it performs in a top-down fashion avoiding bottom-up propagation of larger intermediate results. Its average-case time and worst-case space requirements are estimated and shown to be superior to those of its predecessors by magnitudes.